Skip to content

experiment: test 12x runners on CI workflow - #1667

Closed
lawrencecchen wants to merge 1 commit into
mainfrom
experiment-12x-ci
Closed

experiment: test 12x runners on CI workflow#1667
lawrencecchen wants to merge 1 commit into
mainfrom
experiment-12x-ci

Conversation

@lawrencecchen

@lawrencecchen lawrencecchen commented Mar 18, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Contributor

Testing warp-macos-15-arm64-12x (12 vCPU, 44 GB RAM, 270 GB SSD, $0.16/min) vs current 6x (6 vCPU, 22 GB RAM, 120 GB SSD, $0.08/min) on the CI workflow.

Baseline on 6x (avg of 5 recent runs):

  • tests job: 6m41s ($0.53/run)
  • tests-build-and-lag job: 4m34s ($0.37/run)

Break-even point: if 12x cuts time by >50%, it's cheaper per run despite 2x $/min. Xcode builds are heavily parallelizable so this is plausible.

What to compare after this runs:

  • Job durations on 12x vs baseline above
  • WarpBuild observability: CPU/disk/memory utilization (should drop significantly with 270 GB disk and 12 cores)

Do not merge. Experiment only.

Summary by cubic

Run CI tests and tests-build-and-lag on warp-macos-15-arm64-12x to measure speed vs cost versus 6x; break-even is ≥50% faster given 2x price/min. Updated the CI guard to allow 6x or 12x while keeping the fork billing guard.

  • Refactors
    • Switched both jobs to runs-on: warp-macos-15-arm64-12x.
    • Updated tests/test_ci_self_hosted_guard.sh to accept warp-macos-15-arm64-(6|12)x and simplified error messages.

Written for commit e24b66b. Summary will update on new commits.

Summary by CodeRabbit

  • Chores
    • Updated CI infrastructure with improved runner configurations for enhanced test execution performance and compatibility.

@lawrencecchen
experiment: test 12x runners on CI workflow
e24b66b 
Upgrade tests and tests-build-and-lag jobs from warp-macos-15-arm64-6x
(6 vCPU, 22 GB, 120 GB disk, $0.08/min) to warp-macos-15-arm64-12x
(12 vCPU, 44 GB, 270 GB disk, $0.16/min) to measure speed vs cost tradeoff.

Baseline on 6x (avg of 5 runs):
- tests: ~6m41s ($0.53)
- tests-build-and-lag: ~4m34s ($0.37)

If 12x cuts time by >50%, it's cheaper per run despite 2x price/min.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Mar 18, 2026

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@vercel

vercel Bot commented Mar 18, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cmux Ready Ready Preview, Comment Mar 18, 2026 4:03am

greptile-apps[bot]
greptile-apps Bot reviewed Mar 18, 2026
View reviewed changes

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@coderabbitai

coderabbitai Bot commented Mar 18, 2026 •
edited
Loading

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0ba80fc6-7ce1-4d7f-bb8a-a4ff40c84f21

📥 Commits

Reviewing files that changed from the base of the PR and between 3bca43d and e24b66b.

📒 Files selected for processing (2)
  • .github/workflows/ci.yml
  • tests/test_ci_self_hosted_guard.sh
📝 Walkthrough

Walkthrough

The changes upgrade the CI workflow to use a more capable WarpBuild macOS ARM64 runner (12x instead of 6x) and update the corresponding validation script to accept both runner variants (6x and 12x).

Changes

Cohort / File(s) Summary
CI Workflow Configuration
.github/workflows/ci.yml		 Updated runner specifications from warp-macos-15-arm64-6x to warp-macos-15-arm64-12x in tests and tests-build-and-lag jobs.
CI Guard Validation
tests/test_ci_self_hosted_guard.sh		 Expanded regex pattern to accept both 6x and 12x runner variants; updated failure messages to reference generic warp-macos runner instead of specific 15-arm64-6x variant.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

Poem

🐰 The runners now grow stronger, twelve times might,
From six to double, we ascend new height,
The guard script nods, accepting both with grace,
Warp builds leap faster in the CI race! ⚡

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: testing 12x runners on the CI workflow, which matches the core objective of experimenting with a more capable WarpBuild macOS ARM64 runner.
Description check ✅ Passed The description includes most required sections: summary (what changed and why), testing approach, and notes. However, it lacks explicit testing steps, a demo video section is N/A, and the checklist items are not addressed.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch experiment-12x-ci
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

cubic-dev-ai[bot]
cubic-dev-ai Bot reviewed Mar 18, 2026
View reviewed changes

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

@lawrencecchen

lawrencecchen commented Mar 18, 2026

Copy link
Copy Markdown
Contributor Author

Experiment complete. 12x runners only 7-15% faster but 2x cost/min, not worth it. Staying on 6x.

@lawrencecchen
lawrencecchen deleted the experiment-12x-ci branch March 18, 2026 04:14

This branch was successfully deployed

1 active deployment
Preview — e24b66ba Deployed Mar 18, 2026 by vercel[bot]
View deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cubic-dev-ai cubic-dev-ai[bot] cubic-dev-ai[bot] left review comments
+1 more reviewer
@greptile-apps greptile-apps[bot] greptile-apps[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@lawrencecchen